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DIGITAL IMAGE CROSS TALK CORRECTION 

5 

Background Of The Invention 

1. Technical Field. 

The invention relates to image processing and, more particularly, to cross talk correction 
10 within digital image processing schemes. 

2. Related Art. 

Many conventional methods that are Reared to perform cross talk correction within digital 
image processing schemes commonly employ a\single pass method over the entire digital image 

15 that inherently does not provide a high degree of accuracy. The cross talk correction often does 
converge in a highly accurate manner, given the uke of only a single iteration of cross talk 
correction. Those conventional methods that seek to employ multi-pass cross talk correction 
commonly employ a frame buffer for intermediate data storage. This manner of performing cross 
talk correction in a multi-pass technique is radically expensive^ terms of memory requirements. 

20 That is to say, the entire image needs to be stored as an intermediary to perform a multi-pass 
technique. For example, a single image can contain over 1 million bytes of memory for a SXGA 
image. These processing requirements commonly lead design engineers to generate cross talk 
correction methods that provide less than full correction. Given the enormous memory 
requirements, designers are often bound to solutions that do not provide fulPcross talk correction, 
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at least not to convergence of whateW correction method is employed, within image data 
processing systems that do not have extremely large hardware budget. 

In addition to tho^e methods that inherently require enormous memory requirements, 
other conventional methodsyiave sought to perform cross talk correction in a predetermined 
sequence. For example, in an RGB (red, green, blue) image, the red pixels are corrected first, the 
green pixels are corrected next, md finally the blue pixels are corrected. However, some of the 
deficiencies include significant irregularities and uneven distribution over all of the pixel colors 
within the image. A large amount of image processing is performed on some of the pixels within 
the digital image, while little to no inWe processing is performed on other pixels within the 
same digital image. These irregularities commonly result in high cost and difficulty in 
implementation in hardware. The amount of redesigning and debugging required to 
accommodate all of the various and different digital image types may be enormous. For 
example, to accommodate one digital image type\ the designer must specifically design a method 
adaptable to that image type. Similarly, to accommodate another digital image type, the designer 
must specifically design another method adaptable to that image type. There simply lacks the 
ability to adapt such a conventional method universally ro different types of digital images. 

For a clearer understanding of the problem associated with cross talk, the following 
illustration is provided showing the diffusion of light from one pixel into its neighbors. Cross 
talk between neighboring pixels in a digitalynage occurs when a beam of light aimed for a pixel 
diffuses into its neighboring pixels and corrupts their values. Using the following image pattern 
for illustration, when light is aimed at pixel Bl, \ majority of the light aimed at the pixel may in 
fact captured by the pixel Bl. \ 
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Line 2 G3m B lm G4m B2m . . . 

Line 3 R3m G5m R4m G6m . . . 

Line 4 G7m B3m G8m B4m . . . 

5 As mentioned above, there simply does not exist a highly efficient method to perform digital 
image processing that reduces the undesirable cross talk diffusion between neighboring pixels. 
The image captured inherently contains the embedded cross talk resulting from the imperfections 
within the image sensor itself, in that, the pixels of adjacent pixels are not perfectly isolated from 
one another. Further limitations and disadvantages of conventional and traditional systems will 
10 become apparent to one of skill in the art through comparison of such systems with the invention 
as set forth in the remainder of the present application with reference to the drawings. 
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Line 1 



Rl Gl R2 G2 ... 



Line 2 



G3 Bl G4 B2... 



Line 3 



R3 G5 R4 G6... 



Line 4 



G7 B3 G8 B4... 



5 



However, some of the light escapes into the neighboring pixels Rl, Gl, R2, G3, G4, R3, G5, and 
R4. This undesirable diffusion is color dependent and can be modeled by the following 
equations for each pixel color: 

10 

Blm = bl*Bl + b2*Gl + b3*G5 + M*G3 + b5*G4 + b6*Rl + b7*R2 + b8*R3 + b9*R4 

R4m = rl*R4 + r2*G4 + r3*G5 + r4*G6 + r5*G8 + r6*Bl + r7*B2 + r8*B3 + r9*B4 

15 G4m = gl*G4 + g2*Gl + g3*G2 + g4*G5 + g5*G6 + g6*Bl + g7*B2 + g8*R2 + g9*R4 

where the left hand side of the preceding equations are the cross talk corrupted pixel values and 
the coefficients preceding the pixel values on the right hand side (bl ... b9, rl ... r9, and gl ... 
g9) represent the diffusion coefficients between the adjacent pixels. The actual values captured 
20 by the digital image sensor are corrupted and are represented as follows: 



Line 1 



RlmGlmR2m G2m ... 
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Summary 



Various aspects of the invention can be found in an image processing system operable to 
perform cross talk correction to a digital image having a plurality of pixels. The image 
processing system may include, among other things, an image sensor circuitry, a processing 
5 circuitry, and a line buffer circuitry. The image sensor circuitry typically operates to read an 
image and to generate a digital image. The processing circuitry is communicatively coupled to 
the image sensor circuitry, and the line buffer circuitry, that itself contains a number of line 
buffers that are communicatively coupled to the processing circuitry. 

The processing circuitry performs cross talk correction on a first pixel contained within a 
10 first line of image data. The first line is received from a first line buffer contained within the line 
buffer circuitry. In addition, the processing circuitry uses a second pixel contained within a 
second line of image data to perform the cross talk correction on the first pixel. The second line 
is received from a second line buffer contained within the line buffer circuitry. Also, the 
processing circuitry receives a third line comprising a third plurality of pixels from the image 
1 5 sensor circuitry. 

The processing circuitry may also perform cross talk correction on a third pixel contained 
within the third line of image data after performing cross talk correction on the first pixel. The 
processing circuitry uses a fourth pixel contained within a fourth line of image data to perform 
the cross talk correction on the first pixel to generate the first cross talk corrected pixel, the 
20 fourth line is received from a third line buffer contained within the line buffer circuitry. A 
portion of the second line and a portion of the fourth line are a cross talk correction grid. A 
predetermined trajectory aligns along the first pixel, the second pixel, and the third pixel within 
the image data. The processing circuitry is operable to perform multi-pass cross talk correction 
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on a fourth pixel contained within a fourth line of image data that is received from a fourth line 
buffer contained within the line buffer circuitry. The first pixel, the second pixel, a third pixel 
within the third line of image data, and the fourth pixel are aligned along a predetermined 
trajectory within the digital image. The processing circuitry uses the first cross talk corrected 
5 pixel to perform multi-pass cross talk correction on the fourth pixel. 

This invention may also apply to image processing systems performing cross talk 
correction to a digital image having a plurality of pixels. The image processing system may also 
include, a processing circuitry and a line buffer circuitry. The line buffer circuitry may be 
communicatively coupled to the processing circuitry. The circuitry performs cross talk 
10 correction on a first pixel contained within a first line of image data while the processing 
circuitry receives a second line of image data. The processing circuitry uses a second pixel 
contained within the second line of image data to perform the cross talk correction on the first 
pixel. 

The processing circuitry receives a third line comprising a third plurality of pixels from 
15 the line buffer circuitry. The line buffer circuitry may include a line buffer, and the second line 
of image data is contained within the line buffer. The processing circuitry performs cross talk 
correction on the first pixel contained within the first line of image data while the processing 
circuitry performs multi-pass cross talk correction on a third pixel contained within a third line of 
image data while the processing circuitry receives the second line of image data from the line 
20 buffer circuitry. The first pixel and the third pixel may be aligned along a predetermined 
trajectory within the digital image. A portion of the second line and a portion of the fourth line 
are a cross talk correction grid. The processing circuitry uses the first cross talk corrected pixel 
to perform multi-pass cross talk correction on the third pixel. The processing circuitry uses a 




PATENT 
00CXT0340I 



third pixel contained within a third line of image data to perform the cross talk correction on the 
first pixel. The line buffer circuitry comprises a first line buffer and a second line buffer, and the 
second line of image data is contained within the first line buffer, and the third line of image data 
is contained within the second line buffer. 
5 Other aspects of the invention can be found in a method to perform cross talk correction 

to a digital image. The method includes, among other things, performing cross talk correction on 
a first pixel contained within a first line of image data to generate a first cross talk corrected 
pixel, receiving a second line of image data while performing the cross talk correction on the first 
pixel, and using a second pixel contained within the second line to perform the cross talk 

10 correction on the first pixel. 

In certain embodiments of the invention, the method also includes using a third pixel 
contained within a third line of image data to perform the cross talk correction on the first pixel 
to generate the first cross talk corrected pixel. A portion of the second line and a portion of the 
third line are a cross talk correction grid. The method also includes performing multi-pass cross 

15 talk correction on a third pixel contained within a third line of image data using the first cross 
talk corrected pixel. The method also includes storing the first cross talk corrected pixel in a 
memory location. Also, the first pixel and the second pixel are aligned along a predetermined 
trajectory within the digital image. 

Other systems, methods, features and advantages of the invention will be or will become 

20 apparent to one with skill in the art upon examination of the following figures and detailed 
description. It is intended that all such additional systems, methods, features and advantages be 
included within this description, be within the scope of the invention, and be protected by the 
accompanying claims. 
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Brief Description Of The Figures 

The components in the figures are not necessarily to scale, emphasis instead being 
placed upon illustrating the principles of the\invention. In the figures, likes reference numerals 
designate corresponding parts throughout the different views. 
5 Fig. 1 is a system diagram illustrating an embodiment of an image processing system. 

Fig. 2A is a system diagram illustrating another embodiment of an image processing 
system at a particular instant in time. 

Fig. 2B is a system diagram illustrating the embodiment of the image processing system 
shown in Fig. 2A at another instant in time. 
10 Fig. 3 A is a system diagram illustrating another embodiment of an image processing 

system at another instant in time. 

Fig. 3B is a system diagram illustrating the embodiment of the image processing system 
shown in Fig. 3 A at another instant in time. 

Fig. 4A is a digital image diagram illustrating an embodiment of image processing 
15 performed along a predetermined trajectory by an image processing system. 

Fig. 4B is a digital image diagram illustrating another embodiment of image processing 
performed along another predetermined trajectory by an image processing system. 

Fig. 4C is a digital image diagram illustrating another embodiment of image processing 
performed along yet another predetermined trajectory by an image processing system. 
20 Fig. 4D is a digital image diagram illustrating various embodiments of grid sizes that are 

used to perform cross talk correction, independent of the number of passes performed for cross 
talk correction. 
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Fig. 5A is a digital image diagram illustrating a cross section of a digital image on which 
single pass cross talk correction is performed when a specific pixel within the digital image is 
read. 

Fig. 5B is a digital image diagram illustrating the cross section of the digital image on 
5 which single pass cross talk correction is performed as shown in the Fig. 5A when a subsequent 
pixel within the digital image is read. 

Fig. 5C is a digital image diagram illustrating a cross section of a digital image on which 
multi-pass cross talk correction is performed when a specific pixel within the digital image is 
being read. 

10 Fig. 5D is a digital image diagram illustrating the cross section of the digital image on 

which multi-pass cross talk correction is performed as shown in the Fig. 5C when a subsequent 
pixel within the digital image is read. 

Fig. 6 is a functional block diagram illustrating an embodiment of a cross talk correction 
method. 
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Detailed Description 

Cross talk is an undesirable product of the imperfections in the fabrication of image 
sensor circuitries. This deleterious effect cannot be completely corrected or removed since a 
5 certain degree of information is lost in the image capturing process. Some aspects of this cross 
talk problem are similar to the problem of de-blurring a low-pass filtered image. However, a 
very good estimate of the actual image that is being captured is achieved by performing cross talk 
correction in accordance with the invention. For example, there is virtually no visually 
perceptual difference between the actual image and the cross talk corrected image resulting from 
10 the image processing of the invention. 

To calculate, mathematically, the original (uncorrupted) pixels values of a digital image, 
the diffusion equations for the center pixel value are solved with the following resultant equation. 
The actual values captured by the digital image sensor are corrupted and are represented as 
follows: 

15 

Line 1 Rim Glm R2m G2m ... 

Line 2 G3m B lm G4m B2m . . . 

Line 3 R3m G5m R4m G6m . . . 

Line 4 G7m B3m G8m B4m . . . 

20 
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However, the true value of the pixel B 1 is shown below as a function of the measured 
cross talk corrupted value Blm itself and adjacent pixels. 

Bl = (Blm - b2*Gl - b3*G5 - b4*G3 - b5*G4 - b6*Rl - b7*R2 - b8*R3 - b9*R4)/bl 

Bl = blm*Blm - b2m*Gl - b3m*G5 - b4m*G3 - b5m*G4 - b6m*Rl - b7m*R2 - b8m*R3 - 

b9m*R4 

The actual value of the pixel location Bl is unknown, given the imperfection of the image 
capture device itself (shown as an image sensor circuitry in the various embodiments of the 
invention following). However, recursive mathematics are employed to approximate the actual 
values as shown below. The right hand side of the equation (Gl, G5, ...) is unknown, so those 
values are approximated and replaced with their measured and corrupted values as shown below. 

Blp = blm*Blm - b2m*Glm - b3m*G5m - Mm*G3m - b5m*G4m - b6m*Rlm - b7m*R2m - 
b8m*R3m - b9m*R4m 

Blp is partially cross talk corrected and is closer to the real value Bl than Blm is to Bl. 
Continuing on, doing the same thing to all pixels m the image then we get a partially corrected 
image that is closer to the uncorrupted one. Since ail the pixels were processed only once, this 
particular embodiment illustrates single pass cross talk Correction. If desired, all of the partially 
corrected pixel values are stored in a frame buffer, theri the cross talk correction is repeated 
indefinitely until convergence, or a predetermined number erf time, to generate a higher quality 
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-\(\ image. However, after several passes the cross talk correction converges and further passes will 
not improve or modify the image in any visually perceptual manner. 




This single pass cross talk correction does not require a frame buffer at all. To correct a 
5 pixel, a predetermined number (N) of adjacent rows of the image are used. A number less than 
the predetermined number (N-l) of lines of the image are saved in line buffers since previous 
lines of the image are corrected as a current line comes in from an image sensor circuitry. For 
example, in the instance of using a 3x3 grid size to perform cross talk correction using the single 
pass solution, the amount of storage is only 2x1024 = 2048 pixels - a reduction in memory 
3 10 storage requirements when compared to conventional solutions that perform cross talk 
1 correction. 

3 

* In addition, multi-pass implementations may be performed where certain pixels along a 

j predetermined trajectory are cross talk corrected more than once during a single pass over the 

j image. For example, in the single pass solution, partially cross talk corrected pixel values are 

riJ 15 sent to the next block for further processing or stored in a separate storage like a frame buffer in 

J:f certain embodiments of the invention. However, these same partially cross talk corrected pixels 

v. 5 

are stored back to the line buffers as they are calculated and are used to perform cross talk 
correction on subsequent pixels, thereby giving even greater accuracy. This generates the 
following snapshot of the line buffers as pixel R4 comes in from the sensor when one 
20 predetermined trajectory is employed. 



Line buffer 1 
Line buffer 2 



RlpGlpR2p G2p ... 

G3pBlm G4m B2m 
12 
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Current Line from sensor R3m G5m R4m 



Rip ... G3p are the partially corrected values and Blm ... R4m are the uncorrected values. The 
following equation results for Bl. 

5 

Blp = bl*Blm - b2*Glp - b3*G5 - b4*G3p - b5*G4 - b6*Rlp - b7*R2p - b8*R3 - b9*R4 



This above equation is a better partially corrected Blp since other partially corrected 
neighboring pixels are used in calculating the partially corrected value of Blp. Therefore, this 
10 "single pass" method is superior to the single pass solution discussed above, though it is not truly 
single pass, in that, the pixels in the image are being cross talk corrected multiple times during 
the one pass over the image. Continuing, by adding another line buffer, and when B4m comes 
in from the image sensor circuitry, the following snapshot results. 



15 Line buffer 1 Rip Glp R2p G2p . . . 

Line buffer 2 G3p B lp G4p B2p . . . 

Line buffer 3 R3p G5p R4m G6m . . 

Current Line from sensor G7m B3m G8m B4m 



20 At this point, R4m is partially cross talk corrected and stored as R4p in its place so the 

snapshot will change to the following when the next pixel G9m comes in from the image sensor 
circuitry. 
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Line buffer 1 Rip Glp R2p G2p . . . 

Line buffer 2 G3p B Ip G4p B2p . . . 

Line buffer 3 R3p G5p R4p G6m . . . 

Current Line from sensor G7m B3m G8m B4m G9m 

5 

In this specific embodiment, a complete 3x3 grid of partially corrected pixels (Rip, Glp, 
R2p, G3p, Blp, G4p, R3p, G5p, R4p) is generated. Furthermore, the accuracy of the value of 
Blp is increased by applying the cross talk correction equation to it. When this process is 
continued for all the pixels of the image, then a "second pass" is effectively performed over the 
Q 10 entire image. Therefore, by adding a single line buffer to the image processing system, the 

y:3 

capability of running a second pass over the image is provided without the use of an expensive 
~* frame buffer. 



Similarly, additional line buffers are added in certain embodiments of the invention to 
J 15 help perform more "passes" over the image and the cross talk corrected quality of the image is 
improved even more. This cross talk correction generates high quality image for the same 
number of passes. It is substantially cheaper than conventional cross talk correction methods in 
many respects, one being that it uses less memory (e.g., a few line buffers of 1024 pixels each 
instead of a frame buffer of size 1.3 million pixels). This technique is extendable to grid sizes of 
20 any various size. For example, the above illustration shows a 3x3 diffusion model. However, 
bigger models (5x5 or bigger) are equally operable in accordance with the invention. 

Fig. 1 is a system diagram illustrating an embodiment of an image processing system 100 



fi 



having a processing circuitry 112 and a line buffer circuitry 114. The line buffer circuitry may 
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have an indefinite number of line buffers illustrated as a line buffer A 116, a line buffer B 117, 
and a line buffer N 119. Input, cross talk corrupted image data 120 is fed into the image 
processing circuitry 110. The processing circuitry 112 may be operable to perform cross talk 
correction using any of the embodiments including single or multi-pass cross talk correction and 
5 the use of variable grid sizes for cross talk correction when performing image processing on the 
cross talk corrupted image data 120. The resultant output from the image processing circuitry 
110 is an output, cross talk corrected image data 130. This image data is operable to be stored, 
transmitted or displayed as will be shown in the various embodiments of the invention below. 

Fig. 2A is a system diagram illustrating another embodiment of an image processing 
10 system 200 at a particular instant in time. The image processing system 200 may have an image 
ri sensor circuitry 250, a line buffer circuitry 214, and a processing circuitry 212. The image sensor 
Kl circuitry 250 may be operable to read an indefinite number of lines in an image as shown by a 

r;i 

FIJ line A 251, a line B 252, a line C 253, and a line D 254. The line buffer circuitry 214 itself 

H 

l A contains, at least, a line buffer A 216 and a line buffer B 217. 

pjl 15 A single line (line C 253 in this Figure) is provided to the processing circuitry 212. This 

£3 line is presently the raw image data that is read by the image sensor circuitry 250 and itself 

O 

probably contains cross talk corruption from adjacent pixels. The processing circuitry 212 feeds 

a previously read line of raw image data into the line buffer B 217. The previously read line of 

raw image data in this Figure is shown to be the line B 252. A previously read and corrected line 

20 of image data is fed into the line buffer A 216. This cross talk corrected line is shown to be the 

line A 251. In the event that the current line read is the first line of the image (e.g., the line A 

251), then the processing circuitry simply loads zeroes into the line buffer A 216. The invention 

is operable to perform cross talk correction even when there is no data provided from preceding 

15 
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or adjacent lines and pixels without suffering any catastrophic, operational, or mathematical 
errors. Boundaries conditions (i.e., edge of an image) are handled identically except zeroes are 
loaded into the line buffers instead of preceding or adjacent line and pixel values. The 
processing circuitry 212 is operable to perform mathematical processing on the line B 252 using 
5 the cross talk corrected image data from the line A 251. The processing circuitry 212 is operable 
to perform basic mathematical operations such as multiplication, addition, division, subtraction, 
as well as linear algebra operations which are used in certain embodiments of the invention when 
multi-pass cross talk correction and large grid sizes are used as will be shown below. 

Fig. 2B is a system diagram illustrating the embodiment of the image processing system 
10 200 shown within the Fig. 2 A at another instant in time. Similar to the Fig. 2 A, the image 
processing system 200 may have an image sensor circuitry 250, a line buffer circuitry 214, and a 
processing circuitry 212. The image sensor circuitry 250 may be operable to read an indefinite 
number of lines in an image as shown by a line A 251, a line B 252, a line C 253, and a line D 
254. The line buffer circuitry 214 itself contains, at least, a line buffer A 216 and a line buffer B 
* 15 217. 

A single line (line D 254 in this Figure) is provided to the processing circuitry 212. This 

line is presently the raw image data that is read by the image sensor circuitry 250 and itself 

probably contains cross talk corruption from adjacent pixels. The processing circuitry 212 feeds 

a previously line of raw image data into the line buffer B 217. The previously read line of raw 

20 image data in this Figure is the line C 253 that is read in the Fig. 2A. A previously read and 

corrected line of image data is fed into the line buffer A 216. This cross talk corrected line is the 

line B 252 that is corrected in the Fig. 2A. The processing circuitry 212 is operable to perform 

mathematical processing on the line C 253 using the cross talk corrected image data from the line 

16 
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B 252 in this single pass embodiment of the invention. The processing circuitry 212 may be 
operable to perform basic mathematical operations such as multiplication, addition, division, 
subtraction, as well as linear algebra operations that are used in certain embodiments of the 
invention when multi-pass cross talk correction and large grid sizes are used. 
5 Fig. 3A is a system diagram illustrating another embodiment of an image processing 

system 300 at another instant in time. The image processing system 300 may have an image 
sensor circuitry 350, a line buffer circuitry 314, and a processing circuitry 312. The image sensor 
circuitry 350 is operable to read an indefinite number of lines in an image as shown by a line A 
351, a line B 352, a line R 354, a line S 355, a line T 356, a line U 357, and a line Z 359. 
10 The line buffer circuitry 314 may also contain an indefinite number of line buffers shown as line 
buffer A 316, a line buffer N-l 318, and a line buffer N 319. 

This line is presently the raw image crata that is read by the image sensor circuitry 350 and 
probably contains cross talk corruption from adjacent pixels. The processing circuitry 312 feeds 
a previously line of raw image data into the line\buffer N 319. A previously read and one time 
15 cross talk corrected line of image data in this Figure is shown to be fed into the line buffer N-l 
318. Similarly, previously read and N times cross talk corrected line of image data in this Figure 
is shown to be fed into the line buffer A 316. Any vanable grid size is employed to perform the 
correction of cross talk, as will be shown in even greater qetail in the following Figures. 

The multi-pass cross talk correctioir\is employed such that a one time cross talk corrected 
20 line is located in the line buffer N-l 318, and an N times cross talk corrected line is located in the 
line buffer A 316. The multi-pass solution is operable such that when one pixel is cross talk 
corrected, additional pixels along a predetermined\rajectory within the digital image are also 
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operable to be cross talk corrected afc well. These additional pixels along the predetermined 
trajectory are most likely being cross talk corrected a 2 nd , 3 rd , etc. time, i.e., an Nth time. 

The processing circuitry 312 is operable to perform mathematical processing on a line 

V contained within the line buffer N 319 in this multi-pass embodiment of the invention. The 
5 processing circuitry 312 is operable to perform basic mathematical operations such as 
multiplication, addition, division, subtraction, Us well as linear algebra operations employed to 
perform multi-pass cross talk correction and variable grid sizes. 

Fig. 3B is a system diagram illustrating theVmbodiment of the image processing system 
300 shown in Fig. 3A at another instant in time. Similar to the Fig. 3A, the image processing 

10 system 300 may have an image sensor circuitry 350, a^ine buffer circuitry 314, and a processing 
circuitry 312. The image sensor circuitry 350 is operabld to read an indefinite number of lines in 
an image as shown by a line A 351, a line B 352, a line R 354, a line S 355, a line T 356, a 
line U 357, and a line Z 359. The line buffer circuitry 3 14 may contain an indefinite number 
of line buffers shown as line buffer A 3 16, . . . , a line buffer N- 1 3 1 8, . . . , and a line buffer N 3 19. 

15 A single line (now line U 357 in this Figure) is provided to the processing circuitry 312. 

This line is presently the raw image data that is read by the image sensor circuitry 350 and may 
contain cross talk corruption from adjacent pixels. The processing circuitry 312 feeds a 
previously one time cross talk corrected line of raw image data intq the line buffer N-l 318. 
Depending on the grid size, additional one time cross talk corrected line of raw image data into 

20 the line buffers adjacent to the line buffer N-l 318. Previously read and ^ times corrected lines 
of image data are fed into the line buffer A 316. Similarly, depending on the grid size, 
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additional N times cross talk corrected lines of raw image data into the line buffers adjacent to 
the line buffer A 316. 

The processing circuitry 312 is operable to perform mathematical processing on a line of 
the image data in this multi-pass embodiment of the invention. The processing circuitry 312 is 
5 operable to perform basic mathematical operations such as multiplication, addition, division, 
subtraction, as well as linear algebra operations that are used in multi-pass cross talk correction 
and also when large grid sizes are used to perform cross talk correction. 

Fig. 4A is a digital image diagram illustrating an embodiment of image processing 
performed along a predetermined trajectory by an image processing system 405. An image 407 

10 contains an indefinite number of pixels. The grid sizes employed in the Fig. 4 A are exemplary of 
just one of the many various grid sizes that are each operable to perform cross talk correction and 
is also shown in Fig. 4D. An IN PIXEL is input from an image sensor circuitry and an X PIXEL 
(located immediately to the left one pixel and up one pixel from the IN PIXEL) is cross talk 
corrected. In a single pass solution, the X PIXEL is the last pixel corrected when the IN PIXEL 

15 is read. Moreover, when multi-pass solutions are implemented, a Y PIXEL (located immediately 
to the left one pixel and up one pixel from the X PIXEL) is cross talk corrected at least one time 
more than the X PIXEL. For example, in the two pass solution, the Y PIXEL is the last pixel 
corrected when the IN PIXEL is read. 

When multi-pass solutions are implemented, a Z PIXEL (located immediately to the left 

20 one pixel and up one pixel from the Y PIXEL) is cross talk corrected at least one time more than 

the Y PIXEL. In a three pass solution, the Z PIXEL is the last pixel corrected when the IN 

PIXEL is read. Similarly, the invention is indefinitely scaleable to accommodate any number of 

passes in various multi-pass solutions. When multi-pass solutions are implemented, an N PIXEL 
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(along the predetermined trajectory defined along the X PIXEL, the Y PIXEL, and the Z PIXEL) 
is cross talk corrected. In an indefinite N- pass solution, the N PIXEL is the last pixel corrected 
when the ESf PIXEL is read. 

The predetermined trajectory may be employed to perform cross talk correction to each of 
5 the pixels along the predetermined trajectory. Even in the single pass solution, a predetermined 
trajectory exists by the choosing of which preceding pixel is going to be cross talk corrected 
when the IN PIXEL is read. Moreover, different grid sizes are operable while still performing 
cross talk correction along the predetermined trajectory as shown in the Fig. 4A. For example, 
each of the X PIXEL, the Y PIXEL, and the Z PIXEL, is a cluster of pixels in certain 

10 embodiments of the invention, and a specific pixel within each of the clusters when cross talk is 
corrected. That is to say, a predetermined number of lines is used for cross talk correction in 
each of the passes of the multi-pass embodiments of the invention. 

Fig. 4B is a digital image diagram illustrating another embodiment of image processing 
performed along another predetermined trajectory by an image processing system 415. The Fig. 

15 4B contains image 417 with an indefinite number of pixels. Similar to the Fig. 4A, the grid sizes 
employed in the Fig. 4B are exemplary of just one of the many various grid sizes that are each 
operable to perform cross talk correction and is shown in Fig. 4D below. An IN PIXEL is input 
from an image sensor circuitry and an X PIXEL (located to the left two pixels and up one pixel 
from the IN PIXEL) is cross talk corrected. In a single pass solution, the X PIXEL is the last 

20 pixel corrected when the IN PIXEL is read. When multi-pass solutions are implemented, a Y 

PIXEL (located to the left two pixels and up one pixel from the X PIXEL) is cross talk corrected 

at least one time more than the X PIXEL. For example, in the two pass solution, the Y PIXEL is 

the last pixel corrected when the IN PIXEL is read. 

20 
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When multi-pass solutions are implemented, a Z PIXEL (located to the left two pixels 

and up one pixel from the Y PIXEL) is cross talk corrected at least one time more than the Y 

PIXEL. For example, in a three pass solution, the Z PIXEL is the last pixel corrected when the 

IN PIXEL is read. Similarly, the invention is indefinitely scaleable to accommodate any number 

of passes in various multi-pass solutions. When multi-pass solutions are implemented, an N 

PIXEL (along the predetermined trajectory defined along the X PIXEL, the Y PIXEL, and the Z 

PIXEL) is cross talk corrected. For example, in an indefinite N-pass solution, the N PIXEL is the 

last pixel corrected when the IN PIXEL is read. 

If desired, the predetermined trajectory is also adaptable to be along a trajectory of pixels 

O 10 being located immediately to the left one pixel and up two pixels from a preceding pixel along 

the predetermined trajectory. These various examples of the predetermined trajectory are 

included to illustrate the adaptability of the invention to perform cross talk correction of various 

pixels at various times within the image 417. In addition, similar to the description with respect 

to the Fig. 4A, different grid sizes are operable while still performing cross talk correction along 

15 the predetermined trajectory as shown in Fig. 4B. 

Fig. 4C is a digital image diagram illustrating another embodiment of image processing 

performed along yet another predetermined trajectory by an image processing system 425. The 

Fig. 4C contains image 427 with an indefinite number of pixels. Similar to the Fig. 4A and the 

Fig. 4B, the grid sizes employed in the Fig. 4C are exemplary of just one of the many various 

20 grid sizes that are each operable to perform cross talk correction and is also shown in Fig. 4D. 

An IN PIXEL is input from an image sensor circuitry and an X PIXEL (located to the left three 

pixels and up one pixel from the IN PIXEL) is cross talk corrected. In a single pass solution, the 

X PIXEL is the last pixel corrected when the IN PIXEL is read. When multi-pass solutions are 
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implemented, a Y PIXEL (located to the left three pixels and up one pixel from the X PIXEL) is 
cross talk corrected at least one time more than the X PIXEL. For example, in the two pass 
solution, the Y PIXEL is the last pixel corrected when the IN PIXEL is read. 

When multi-pass solutions are implemented, a Z PIXEL (located to the left three pixels 
5 and up one pixel from the Y PIXEL) is cross talk corrected at least one time more than the Y 
PIXEL. For example, in a three pass solution, the Z PIXEL is the last pixel corrected when the 
IN PIXEL is read. Similarly, the invention is indefinitely scaleable to accommodate any number 
of passes in various multi-pass solutions. When multi-pass solutions are implemented, an N 
PIXEL (along the predetermined trajectory defined along the X PIXEL, the Y PIXEL, and the Z 

10 PIXEL) is cross talk corrected. For example, in an indefinite N-pass solution, the N PIXEL is the 
last pixel corrected when the IN PIXEL is read. 

If desired, the predetermined trajectory is also adaptable to be along a trajectory of pixels 
being located immediately to the left one pixel and up three pixels from a preceding pixel along 
the predetermined trajectory. These various examples of the predetermined trajectory are 

15 included to illustrate the adaptability of the invention to perform cross talk correction of various 
pixels at various times within the image 427. In addition, similar to the description of Figs. 4A 
and 4B, different grid sizes may be operable while still performing cross talk correction along the 
predetermined trajectory as shown in Fig. 4C. 

Fig. 4D is a digital image diagram illustrating various embodiments of grid sizes 435 that 

20 are used to perform cross talk correction, independent of the number of passes performed for 

cross talk correction. The number of passes used to perform cross talk correction in the various 

multi-pass embodiments of the invention is independently variable from the number of passes 

performed. Fig. 4D shows the instances when cross talk is not corrected on a pixel, but rather a 
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location between pixels. For example, position A, position C, and position E are all exemplary 
of how it need not be a pixel itself that is being cross talk corrected, but rather a location between 
pixels. 

In the illustration shown in Mg. 4D, position A employs a 2xM grid size to perform cross 
talk correction. A single line buffer may be required to perform this operation. The grid size 
2xM is an indefinitely scaleable rectangle where the height of the rectangle is 2 lines of image 
data. Similarly, position C employs a 4xM\grid size to perform cross talk correction. Three line 
buffers may be required to perform this operation. The grid size 4xM is an indefinitely scaleable 
rectangle where the height of the rectangle is\4 lines of image data. Similarly, position E 
10 employs a 6xM grid size to perform cross talk correction. Five line buffers may be required to 
perform this operation. The grid size 6xM is an indefinitely scaleable rectangle where the height 
of the rectangle may be 6 lines of image data. 

For cross talk correction performed on actual pixels within the image, as shown in Fig. 
4D, the pixel B employs a 3xM grid size to perform cross talk correction. Two line buffers may 
15 be required to perform this operation. The grid size 3xM is an indefinitely scaleable rectangle 
where the height of the rectangle is 3 lines of image data. Similarly, the pixel D employs a 5xM 
grid size to perform cross talk correction. Four line buffers may be required to perform this 
operation. The grid size 5xM may be an indefinitely scaleable rectangle where the height of the 
rectangle is 5 lines of image data. Similarly, the pixel F employs a 7xM grid size to perform 
20 cross talk correction. Six line buffers may be required to perform this operation. The grid size 
7xM is an indefinitely scaleable rectangle where the height of the rectangle is 7 lines of image 
data. 
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Also shown in Fig. 4D, a pixel or position N employs an NxM grid size to perform cross 
talk correction. N-l line buffers are required to perform this operation. The grid size NxM is an 
indefinitely scaleable rectangle where the height of the rectangle is N lines of image data. 
Various grid sizes may be operable with various number of passes performed in the multi-pass 
5 solutions. 

Fig. 5A is a digital image diagram illustrating a cross section snapshot 500 of a digital 
image on which single pass cross talk correction is performed when a specific pixel within the 
digital image is read. In Fig. 5A, when the pixel location R4 is read, it is the pixel location G3 
(shown as the single time cross talk corrected pixel G3pl) that is cross talk corrected. For 

10 example, the measured value (R4m) is read by an image sensor circuitry, and the measured value 
(G3m) contains some undesirable cross talk corruption, the cross talk is corrected resulting in the 
cross talk corrected pixel (G3pl). 

Fig. 5B is a digital image diagram illustrating the cross section snapshot 502 of the digital 
image on which single pass cross talk correction is performed as shown in Fig. 5A when a 

15 subsequent pixel within the digital image is being read. In Fig. 5B, when the pixel location G6 is 
read, it is the pixel location Bl (shown as the single time cross talk corrected pixel Blpl) that is 
cross talk corrected. It is the pixel location G3 (shown as G3pl) that has been cross talk 
corrected in the previous snapshot as shown in Fig. 5A. For example, the measured value (G6m) 
is what is actually being read by an image sensor circuitry, and the measured value (Blm) itself 

20 contains some undesirable cross talk corruption is cross talk corrected. The pixel location Bl, 
having been cross talk corrected, is shown as (Blpl) indicating a single pass cross talk correction 
solution. 
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Fig. 5C is a digital image diagram illustrating a cross section snapshot 504 of a digital 
image on which multi-pass cross talk correction is performed when a specific pixel within the 
digital image is read. In Fig. 5C, when the pixel location B4 is read, it is the pixel location G5 
(shown as the single time cross talk corrected pixel G5pl) that is cross talk corrected. For 
5 example, the measured value (B4m) is read by the image sensor circuitry, and the measured value 
(G5m) contains some undesirable cross talk corruption that is cross talk corrected to become the 
cross talk corrected pixel (G5pl). 

Fig. 5D is a digital image diagram illustrating the cross section snapshot 506 of the digital 
image on which multi-pass cross talk correction is performed as shown in Fig. 5C when a 

10 subsequent pixel within the digital image is read. In Fig. 5D, when the pixel location G9 is read, 
it is the pixel location R4 (shown as the single time cross talk corrected pixel R4pl) that is cross 
talk corrected. It is the pixel location G5 (shown as G5pl) that has been cross talk corrected in 
the previous snapshot as shown in Fig. 5C. For example, the measured value (G9m) is read by 
the image sensor circuitry, and the measured value (R4m) contains some undesirable cross talk 

15 corruption that is cross talk corrected. The pixel location R4, having been cross talk corrected, is 
shown as (R4pl) indicating a single pass cross talk correction solution. 

The various embodiments of the invention shown in Figures 5A, 5B, 5C, and 5D show 
one particular predetermined trajectory where cross talk correction is performed. However, those 
having skill in the art of digital image processing will appreciate that any of the various 

20 predetermined trajectories shown in the various Figures are analogously operable without 
departing from the scope and spirit of the invention. In addition, those predetermined trajectories 
that are not explicitly illustrated are also within the scope of the invention. 
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Fig. 6 is a functional block diagram illustrating an embodiment of a cross talk correction 
method 600. In a block 610, a pixel is received from an image sensor. In a block 620, a 
predetermined number of pixels are retrieved from a predetermined number of line buffers. In 
block 630, cross talk correction is performed on a selected pixel or a selected number of pixels 
5 that is retrieved from a predetermined number of line buffers in the block 620. The cross talk of 
the pixels is corrected along a predetermined trajectory as shown in Figures 4A, 4B, and 4C. In 
addition, the grid size of the cross talk correction is also variable, as shown in Fig. 4D. In a 
block 640, the predetermined number of cross talk corrected/retrieved pixels are updated and 
stored into a predetermined number of line buffers. Within the multi-pass embodiments of the 
q 10 invention, certain pixels along the predetermined trajectory are updated 2 nd , 3 rd , 4 th , etc. times, 
0 a§ depending on the number of passes employed in the specific multi-pass embodiment. Those 
2* updated values are then stored into the predetermined number of line buffers, so that the cross 

e Is? 
~; S 

11 talk corrected values are usable when performing cross talk correction on subsequent pixels. The 

£3 ability to use partially cross talk corrected values to perform cross talk correction of other pixels, 

fij 15 allows for a high degree of accuracy when compared to conventional methods of cross talk 



In an alternate block 660, the predetermined number of cross talk corrected/retrieved 
pixels may be stored, transmitted, and/or displayed. For example, the cross talk correction 
method 600 does not require that the entire image be stored in any intermediary memory 
20 location, as compared with conventional methods, but the cross talk correction method 600 is 
operable to allow the cross talk corrected/retrieved pixels to be stored in a location for 
subsequent use. This type of memory is typically cheaper than the dynamic memory required to 
perform full image data processing used in conventional image processing systems. In addition, 
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those cross talk corrected/retrieved pixels are operable to be transmitted via local or wide area 
network. Similarly, the cross talk corrected/retrieved pixels are operable to be displayed using 
any suitable display system. Finally, in block 650, the current pixel, that is received from the 
image sensor in the block 610, is stored into a line buffer for subsequent use by other pixels to 
perform cross talk correction within the digital image. 

While various embodiments of the invention have been described, it will be apparent to 
those of ordinary skill in the art that many more embodiments and implementations are possible 
that are within the scope of this invention. 
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